A decentralized adaptive momentum method for solving a class of min-max optimization problems

Min-max saddle point games have recently been intensely studied, due to their wide range of applications, including training Generative Adversarial Networks (GANs). However, most the recent efforts for solving them are limited special regimes such as convex-concave games. Further, it is customarily assumed that underlying optimization problem solved either by a single machine or in case multiple machines connected centralized fashion, wherein each one communicates with central node. The latter approach becomes challenging, when communications network has low bandwidth. In addition, privacy considerations may dictate certain nodes can communicate subset other nodes. Hence, interest develop methods solve min-max decentralized manner. To end, we adaptive momentum (ADAM)-type algorithm under condition objective function satisfies Minty Variational Inequality condition, which generalization case. proposed method overcomes shortcomings non-adaptive gradient-based algorithms problems do not perform well practice and require careful tuning. this paper, obtain non-asymptotic rates convergence (coined DADAM$^3$) finding (stochastic) first-order Nash equilibrium subsequently evaluate its performance on GANs. extensive empirical evaluation shows DADAM$^3$ outperforms developed methods, optimistic stochastic gradient problems.